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I There are many interesting mathematical problems in control theory. In this paper we 

>• i will discuss problems and techniques related to underactuated systems. An under actuated 

C^ ■ system is one with fewer control inputs than degrees of freedom. Balancing a ruler on the 

(^ ■ tip of a finger is a good example of an underactuated system. This system has five degrees 

-J. ' of freedom (three for the fingertip and two angles for the ruler). However, only the three 

^ I degrees of freedom for the fingertip are directly controlled. In fact, any system requiring 

"^ . balance is an underactuated system. A bicycle is an obvious example. An airplane is a 

rj ■ less obvious example (six degrees of freedom, underactuated by two). 

IJ, ' We will give a mathematical formulation of several problems arising from applications, 

'k>( I review some standard and new techniques, and pose some interesting and challenging 

^ . open questions. 

Stabilization of underactuated systems 

To describe a mechanical system we start with a manifold, Q, representing all possible 
configurations of the system. The configuration space Q is equipped with a Riemannian 
metric, g, so that the kinetic energy is ^g{x, x). It also comes with a function V : Q ^ M., 
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and two fiber preserving maps c, / : TQ — > TQ. The function V represents potential 
energy, c represents dissipation, and / represents applied external forces. The equations 
of motion are given by 

V^7 + c(7)+ grad-yV = fi'j). (1) 

The external forces, /, are used to control the system. The system obtained by setting 
the control input to zero is called the open loop system. Equation (1) including the control 
input is called the closed loop system. The system is underactuated if / is restricted to 
be in some directions. In other words, there is a (/-orthogonal projection P onto the 
subspace of unactuated directions, and P{f) must vanish. 

The basic problem is to find a function / in some class so that solutions to equation 
(1) have some desired properties. Physically one may not always be able to measure the 
full state (x, x) G TQ of the mechanical system. In this situation the function / must 
only depend on the observable variables. For now, we will consider the case when all 
variables may be observed. This is referred to as full state feedback control. 

The stabilization problem is to find a control input, so that some point (xq, 0) will be 
an asymptotically stable equilibrium. Other notions of stability may also be considered, 
however, asymptotic stability is the most useful in applications. We will next review 
several approaches to the stabilization problem. 

The most commonly employed technique used to address this problem is linearization. 
Choosing / so that the eigenvalues of the linearized equation lie in the left half plane, will 
ensure that the desired point is a locally asymptotically stable equilibrium. This reduces 
the problem to an algebraic question that can be easily solved and implemented. 

First, note that it is not possible to stabilize every system, for example, 

x^ - x^ = 

••2 I 2 

X + X = U . 

For a linear n x n system of the form y = Ay + Bu necessary and sufficient conditions 
for the existence of a linear stabilizing control law u = Cy are well known [14, 18]: the 
rank of the matrix [si — A, B] must be n for all Re s > 0. If a linear stabilizing control 
law exists, there is a finite dimensional family of linear stabilizing control laws. 

Once it is known that a stabilizing control law exists, one must choose a specific control 
law. The problem of finding a matrix C given the eigenvalues of A + BC is called pole 

placement. Engineers use various rules of thumb to decide where to place the poles. 
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These rules of thumb are based upon the behavior of solutions to a constant coefficient 
second order ODE. For higher order systems one purposefully places two dominant poles, 
zi and ^2 5 with the remaining poles near the real axis and far to the left. This enables 
one to approximate solutions of the higher order system by solutions of a second order 
system. 

Example 1: The inverted pendulum cart 




X 

Figure 1 

With appropriate scaling the metric g is given hj g = d9^ + 2b cos (6*) dx dO + dx^, where 
6 is a physical parameter, < 6 < 1. The potential energy is given hj V = cos(6'). Since 
no torques can be applied directly to the pendulum, P = {bcos{9) dx + dO) ® d /dO is the 
orthogonal projection onto the direction d/d9. Assuming that there is no dissipation, 
c = 0. The d/d6- and 9/9x-components of the equations of motion read: 

^■ + 6cos(^)x-sin(^) =0 

bcos{e)e + x-hsm{e)e'^ = u, 

where u = g{d/dx, f) represents the external force applied to the base of the cart, and it 
is the control input. 

The linearization around 9 = 0, x = 0, 9 = 0, x = reads 
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where u = ai9 + a2X + a^6 + a4^x is the hnearized control input. The characteristic 
polynomial of the above matrix is 

4 036-04 3 l + a2-ai6 2 , a4(l + 62) 02(1 + 6^) 
-^ ^^r^W^ + 1-62 -^ + (1-62)2-^+ (1-62)2- 

Once the desired eigenvalues are specified, it is an easy matter to solve for the ak and 
get the control law. 

Another standard technique employed in control design is linear quadratic optimal 
control, [18]. For linearized systems y = Ay + Bu, one looks for control laws which will 
minimize the functional 



Jiy,u)= / {y{t),Qyit)) + {uit),Nu{t))dt 
Jo 

subject to the condition y = Ay + Bu. Here Q and A^ are positive definite quadratic 

matrices which are usually specified by engineering rules of thumb. 

If a stabilizing control law exists, the resulting control input can be expressed as a 

linear function of the state y. 

Linearization works very well for many practical applications. Unfortunately, the 
limitations of linearization are seldomly discussed. Continuing in this tradition we will 
now address nonlinear methods without stating why. 

Many nonlinear methods employ the notion of a Lyapunov function. For a system of 
ODEs, X = f{x), a Lyapunov function is a function F{x) which is bounded from below 
and decreases along the trajectories, [4] . It can be defined globally or locally. If xq is a 
stationary solution of i; = f{x), and a unique local minimizer of the Lyapunov function 
-F, then xq is Lyapunov stable. If F is nonconstant along nonstationary trajectories then 
Xq will be locally asymptotically stable. If, in addition, F is defined globally, then xq is 
a global asymptotically stable equilibrium. 

For the stabilization problem, x = f{x,u), one wishes to find a control input, u, as a 
function of the state, x, and a Lyapunov function, -F(x), so that the closed loop system 
admits F{x) as a Lyapunov function. In the linear case, y = Ay + Bu, with u = Cy, 
one looks for a quadratic Lyapunov function F{y) = {y, Ky). The function F will be a 
Lyapunov function if and only if 

D = {A + BCyK + K{A + BC) (2) 
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is negative definite. Given any negative definite D one may solve equation (2) for K if 
and only if all the eigenvalues of A + SC are in the left half-plane. The solution is unique: 

l>oo 

Jo 
Note, that in the optimal control approach discussed previously, the value function 

F{x) = inf J(x, u) 

u 

is a Lyapunov function. We also remark that, if there is a locally asymptotically stabi- 
lizing control law, then there exists a local Lyapunov function, [4]. 

Several recent papers propose to find control inputs so that the closed-loop system 
(1) would have a natural candidate for a Lyapunov function, [1-3, 5-9, 13, 17]. In [3] we 
introduce the following approach to the stabilization problem for underactuated systems. 

We consider the control problem 

V^7 + c(7)+ grad^V = f{^), (1) 

subject to the constraint -P(/) = 0. We wish to find / and a Lyapunov function simulta- 
neously. In fact, we will describe an infinite dimensional family of control inputs. 

Our approach to this question is to find functions g^ c, V and / so that solutions to 
Equation (1) are automatically solutions to 

V^7 + c(7) + 'gmd^V = 0. (3) 

This is the matching philosohy. The motivation for this philosophy is that 

H{X) = ^g{X,X) + V 

is a natural candidate for a Lyapunov function because -^H{X) = —g{c{X), X). A state, 
Xq G TQ will be an asymptotically stable equilibrium if H{X) > 0, and — ^(c~(X), X) > 
with equality only at Xq. 

Equations (1) and (3) clearly hold if and only if: 

f{X) = VxX - VxX + grad^V - ^d^V + c{X) - c(X), (4) 
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for every vector field X. Tfie condition P{f) = tfien becomes a system of nonlinear 
partial differential equations for ^, V , and c. Notice that constant multiples of g, c, and 
V satisfy P{f) = even when P has full rank. Thus, one would expect many solutions 
when P does not have full rank. Separating P{f) = into terms which are quadratic in 
the velocity, independent of the velocity or odd functions of the velocity gives: 

P{VxX-VxX) = 0, (5.1) 

P{grad^V - grad^V) = 0, (5.2) 

P{c{X)-c{X))=0. (5.3) 

We will look for solutions to these matching equations with 'g non-degenerate so that 
g{X, Y) = g{\X, Y) with A G T{T*Q ® TQ). It is clear that A has to be g self-adjoint, 
i.e., g{\X, Y) = g{X, XY). We will derive a linear system of partial differential equations 
for A which must be satisfied if ^ is to solve Equation (5.1). 

In our previous paper, we described a method to find every solution to the matching 
equations by solving three linear systems of partial differential equations in a row. We 
will review this method now. 

One first solves the equations 



V<7A|j^p^, =0, (6) 



for A|im p. Then one solves 



L^Px9 = Lp^g (7) 

(this is a slight rewrite of equation (1.12) of our previous paper [3]), 

L,,,V = L,,V (8) 

(this is equation (1.13) of our previous paper [3]), then after solving equation (5.3), the 

control input will be given by (4). 

In the previous paper we explicitly showed that any solution to the matching equations 

solves equations (6), (7), (8), and (5.3) (Propositions 1.1, 1.2, and 1.3 of [3]). Implicit 

in [3, Proposition 1.4] is the fact that any solution of equations (6), (7), (8) and (5.3) is 

in turn a solution to the matching equation. We will make this argument explicit now. 
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Indeed, taking into account the fact that P is (7- self adjoint, our ^-Equation (7) iniphes 
the matching equation (5.1). Here is a short proof. For any Z and X we have 

g{P{VzZ-VzZ), X) 
=g{VzZ - VzZ, PX) 
^9{VzZ, XPX) - g{VzZ, PX) 

=Zg{Z, XPX)-g{Z, VzXPX) - Zg{Z, PX) + g{Z, VzPX) 
^-g{Z, VxpxZ)+g{Z, [XPX, Z]) + g{Z, VpxZ)-g{Z, [PX,Z]) 

^-^{L,,,g{Z,Z)) + ^{L,,g{Z,Z)) 
=0. 

Since this is true for all X, P{VzZ - VzZ) = 0. 

Equations (6) and (7) imply additional compatibility conditions. Even though we do 
not know all the compatibility conditions in general, we do know all the compatibility 
conditions for systems with two degrees of freedom. Let us summarize our method in the 
case of two degrees of freedom one of which is unactuated. Since the unactuated subspace 
is one dimensional, it can be locally expressed as the span of a unit length vectorfield, 
PX. Choose coordinates x^, x"^ so that PX = -^. In these coordinates gn = 1. We will 

may be rewritten as 



always write XPX = ctj^y + fJ'jr^i where a and n are yet to be found. The A-equation 



Q^(9iicr + gi2^i) - 2[11, 2] // = 0, ^(^n^ + ^i2/«) - 2[12, 2] // = 0. (9) 

Here, 

[u, k] = giVd.dj, dk) = -{gikj + gjk,i - gij,k)- (10) 

For these equations to be consistent the following compatibility condition must hold: 

^■([11,2]^) = A([i2,2]^). (11) 



dx"^ ' dx^ 

Notice that this is a first order partial differential equation equation. Theoretically, it 
can be solved for ^ via the method of characteristics. Generically, a solution will include 
an arbitrary function of a single variable. Once ^ is known, a is given by 

a{x^,x'^) = gi2{x'^,x'^)n{x'^,x'^) +2 / {[11,2] n{x'^,x'^) dx^ + [12,2] n{x'^,x'^) dx"^) . 



The next step is to solve equations (6) for g. It turns out that it is easiest to solve 
first for ^n and then find the remaining components from the algebraic system 

9 = 9^- 
First note that the {11} component of the right side of (7) is 



' dx^ ' dx^ 



dx^ 



Next, 



iLxPx9){^, i) = APX(^n) - 2g{[XPX, ^ ^ ^ 



Since 



[A^^^ 7^: 



' dx^ dx^ 
da d dfi d 



dx^ ' dx^ 



^_da__d_ ^ o_di^^_d_ _ 1 d^ ^^ .^ 



dx^ dx^ dx^ dx^ dx"^ dx^ dx^ H dx^ dx^ (i dx^ 

and ^A = g, we obtain, 

, „,,^ ^, da a du .^ Ida 

XPXg^^ - 2{~j— + -TTT ^11 + 2-7rT^ii = 0. 
OX^ /U ox^ n ox^ 

Thus, gii satisfies the following first order partial differential equation 



^ ill ^ ^^ ^ 2(^ - --^)gii + 2& = 0. 

■J _L /^nf ^ htnc -f / / hi IT' -f / / 



dx^ 



dx'^ 



' dx^ H dx^ 

The general solution to this first order PDE has an arbitrary function of a single variable 
in it. It is, once again, possible to solve this equation by the method of characteristics. 
Denote by y(x^, x^) any solution of the homogeneous equation 

dy dy 







(12) 



dx^ dx"^ 
such that dy/dx'^ ^ 0. Let a and // be a and \x considered as functions of x^ and y, i.e. 

a{x^ ^y{x^ ^x^y) = a{x^ ^x^^^ \i{x^ ^ y{x^,x'^)) = ii{x^ ^x"^). 

Then the solution to equation (7) is given explicitly by 



gii{x ,x ) 



^ 



CT O Li ill 7//1 2\\ 

^77-— dx , 1 ^.+h{y(x ,x )) 

fl^ dx^ \y=y{x\x2) V«V ^^ 



(13) 



where h{y) is an arbitrary function of a single variable. After ^n is found, we have 

912 = -(fi'ii - cr?ii), 922 = -{912 - (^912)- (14) 

The equation for V reads 

dV dV _ dV 

dx^ dx'^ dx^ 
Again, this equation can be solved by the method of characteristics. Once a solution y 
of (12) is known, V is given by: 

V{x^,x'^) = - -^dx^\y^y^^,^^^^+w{y{x^,x'^)), (15) 

where V^i{x^,y{x^,x^)) = ^^^qJ"" ^ Also, 

Cl =Ci +912{C2 -C2). 

It is convenient to write the non-zero control input as a sum: 

d - -^ - d 

u = 9{f^ o"^) = 9{'^xX-VxX+ grad-yV-grad^V + c{X)-c{X), -—^) = Ug + uy+Uc, 

(16) 
where 

d — - a d 

Ug = g{VxX-VxX,-—), uv = 9i9rad^V -grad^V, -—^), Uc = g{c{X)-~c{X),-—), 

In coordinates, 

Ug = {[i3,2\-gk2V%)x'x\ (17) 



^^ are the Christoffel symbols corresponding 



^'^P is the inverse matrix to ^irm and [ij^p] is defined as in (9) with all (7's replaced by 
^'s. The next term is: 

uv = V^2 -f^ gj2%i. (18) 

Finally, 

Wc = det5f(c2 -C2). (19) 

From the explicit formulae (16)-(19) one sees that the first order germs of our control 
inputs contain every possible linear control law: 



Theorem. // a system with 2 degrees of freedom underactuated by 1 is linearly stahiliz- 
able, then, by choosing appropriate solutions of our matching equations, the linearization 
of the controlled system will have prescribed eigenvalues in the left halfplane. Even if the 
system is not stabilizable, any linear control law may be obtained as the first order germ 
of some control law in our family. 

Open problems 

One of the reasons that the limitations of linearization are seldomly discussed is that 
there is no effective criteria to compare arbitrary control laws. To be more specific we 
will restrict our discussion to the stabilization problem. 

A good stabilizing control law will produce a large basin of attraction, send solutions 
to the equilibrium in a short period of time, and will have low cost. Of course, the precise 
meaning of "large" , "short" , and "low" depends on the concrete engineering problem. It 
is not clear how to quantify these concepts. For linear systems the size of the basin 
of attraction is irrelevant since the whole space is the basin of attraction of a stable 
equilibrium. For nonlinear systems this question is subtle. One could just use the volume 
or diameter as a measure of the size of the basin of attraction. These are not, however, 
usually appropriate measures of size, see Figure 2. In addition, they are difficult to 
compute. 





Figure 2 

Alternatively, one could measure the radius of the largest inscribed ball centered at 
the equilibrium. It is not, however, clear which metric should be used in state space. It 
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is difficult to analytically or numerically estimate this radius. Even if one could compute 
this radius, it might not be the most relevant measure of performance. Real systems have 
an operating range: roUerblades are not designed to handle Mach 2. So, let B be the 
basin of attraction and O be the operating range. It is more reasonable to measure the 
size of the subset M d B consisting of all states whose forward trajectories remain in the 
operating range, see Figure 3. We will call M the normal operating range. It is just as 
difficult to form a reasonable measure of the size of M . 




Figure 3 

Problem. Find an effective analytical or numerical method to estimate the injectivity 
radius of the normal operating range or define a better measure of "size" together with 
an effective analytical or numerical estimate. 



In engineering applications the time it takes to drive the states to the equilibrium is 
very important. This is usually characterized by the rise time, peak time, settling time, 
etc., [11, p. 222]. These notions are only well defined for solutions of mz + cz + kz = f 
with m, c, k, and / constants. Considering the initial conditions with i = only, one 
rescales the initial value problem to z + 2(^z + z = lj z{0) = 0, i(0) = 0. The rise time, 
Tr, is the first time z{t) reaches 1. The time constant is 1/^. The settling time is the 
time it takes z{t) to get and stay within 2% of the final value 1. These notions are used 
for linear constant coefficient ODEs which have two dominant poles. 

One can define analogues of these notions for nonlinear systems. An engineer may 

choose a target operating range T>, which is a small neighborhood of the equilibrium. 
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Figure 4 



Define the A/'I^-settling time 



Tmt> = sup {t > 0| x(t; Xq) ^ V} . 

xoEAf 

This is the time after which any trajectory starting in the normal operating range A/" will 
settle into the target range V. 

Problem. Find an effective analytical or numerical method to estimate the MV-settling 
time or define a better measure of ''short time" together with an effective analytical or 
numerical estimate. 

Another important characteristic of a control law is the cost. One often wishes to 
minimize some function of the trajectory and/or the control input (for example, minimize 
the energy used to complete a task). This imposes a non-trivial restriction on a control 
problem. Real life systems have additional physical restrictions. For example, there is a 
maximal voltage which may be applied to a DC motor before it saturates. 

Reiterating, a good stabilizing control law should maximize the basin of attraction and 
minimize the time and cost while operating within the physical restrictions of the given 
system. 

The problem of maximizing the size of the basin of attraction taken to the extreme 
leads to the question of finding the topologically best control laws. For example, the 
state space for the inverted pendulum cart is S"^ x M x M^. For topological reasons it is 
impossible to find a control law so that the resulting flow has a globally asymptotically 
stable equilibrium in this case. The best one could hope for is a flow with a regular 

compact global attractor, [12, 16]. Recall that such an attractor is the union of the 
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unstable manifolds of finitely many hyperbolic fixed points. Also recall that the shape 
(in the sense of K. Borsuk) of the (global) attractor must be the same as the shape of 
the state space, [15]. We wish to minimize the number of fixed points. For the inverted 
pendulum cart the topologically best flow is depicted in Figure 5. 





S - [R - [R 

Figure 5 

This flow must have one index fixed point and one index 1 fixed point. The basin of 
attraction of the index fixed point is the complement of a properly embedded R"^. 

Problem. When is it possible to find a control input producing a topologically best flow? 
Is there a method to find such control laws? 

At the moment we do not even know whether a topologically best control law exists 
for the inverted pendulum cart. Furthermore, we do not know whether a control law with 
a global attractor exists! 

A related problem is to describe the behavior of the system far away from the equilib- 
rium. It is possible to compactify some dynamical systems with algebraic nonlinear ities, 
[10]. The advantage of a compactification is that overall dynamics may be well under- 
stood by linearization about each of the critical submanifolds. The problem is that for 
nonalgebraic nonlinearities it is not known when a compactification exists. It is even 
interesting and instructive to try to compactify the system 9 + c9 + sin(6') = 0. 

We wrote this paper to encourage other people to think about nonlinear control theory. 

It has a great many unanswered fundamental questions. The field is ready for new ideas. 
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